Highlight Ranking for Broadcast Tennis Video Based on Multi-modality Analysis and Relevance Feedback

Most of existing work on sports video analysis concentrates on highlight extraction. Few efforts devoted to the important issue as how to organize the extracted highlights which is adapt for the user preference. In this paper, we propose a novel approach to rank the highlights extracted from broadcast tennis video based on multi-modality analysis and relevance feedback. Firstly, visual and auditory features are employed to construct the mid-level representations for the content of broadcast tennis video. Then, the affective features are extracted from mid-level representations and the multiple ranking models are built using nonlinear regression algorithm. Finally, the ranking models are linearly combined to generate the final highlight ranking results. The relevance feedback technique is employed to effectively capture the user interest in visual and auditory attention spaces to adjust the ranking results being suitable to the user preference. The experimental results are encouraging and demonstrate that our approach is effective.